Graphic interface for real-time vision enhancement

ABSTRACT

Imaging systems can often gather higher quality information about a field of view than the unaided human eye. For example, telescopes may magnify very distant objects, microscopes may magnify very small objects, and high frame-rate cameras may capture fast motion. The present disclosure includes devices and methods that provide real-time vision enhancement without the delay of replaying from storage media. The disclosed devices and methods may include a live view user interface with two or more interactive features or effects that may be controllable in real-time. Specifically, the disclosed devices and methods may include a live view display and image and other information enhancements, which utilize in-line computation and constant control.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

Cameras and their associated optics may capture information from scenesthat is often better than what unaided human eyes can detect. Forexample, telescopes can magnify very distant objects, microscopes canmagnify very small objects, high frame-rate cameras can capture muchfaster motion, and so on. However, most examples are designed to storeimages or video files for replay at a later time. Furthermore, many ofthe examples are not portable and many of them are designed for specificscientific tasks.

The present disclosure provides systems and methods for a real-time,live view display with image and other information enhancements usingin-line computation and constant control. In particular, the disclosuremay include a live view user interface with two or more interactivefeatures or effects, which are executable and controllable in real-time.In other words, the disclosed invention permits real-time visionenhancement without the delay of replay from storage media.

SUMMARY

In a first aspect, a system is provided. The system includes a display,a touch-based control interface, and a control system. The display isconfigured to display a live view representation of a field of viewbased on image data from one or more image-capture devices. The controlsystem is configured to generate the live view representation of thefield of view based on image data from one or more image-capturedevices. The control system is also configured to receive a controlinput for two or more effects via the touch-based control interface.Additionally, the control system is configured to, for each of the twoor more effects, respond to receipt of the control input for the effectvia the touch-based control interface by applying the effect to the liveview representation in real time.

In a second aspect, a method is provided. The method includes providinga live view representation of a field of view to a display. The liveview representation is based on image data from one or moreimage-capture devices. The method also includes receiving a controlinput for two or more effects via a touch-based control interface.Additionally, the method includes, for each of the two or more effects,responding to receipt of the control input for the effect via thetouch-based control interface by applying the effect to the live viewrepresentation in real time.

In a third aspect, a non-transitory computer readable medium isprovided. The non-transitory computer readable medium includesinstructions executable by a computing device to cause the computingdevice to perform functions, the functions including causing a live viewrepresentation of a field of view to be provided to a display. The liveview representation is based on image data from one or moreimage-capture devices. The functions further include receiving a controlinput for two or more effects via a touch-based control interface. Thefunctions yet further include, for each of the two or more effects,responding to receipt of the control input for the effect via thetouch-based control interface by causing the effect to be applied to thelive view representation in real time.

These as well as other aspects, advantages, and alternatives, willbecome apparent to those of ordinary skill in the art by reading thefollowing detailed description, with reference where appropriate to theaccompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a perspective view of an example system, according to anillustrative embodiment.

FIG. 1B is a perspective view of an example system, according to anillustrative embodiment.

FIG. 1C is a perspective view of an example system, according to anillustrative embodiment.

FIG. 1D is a perspective view of an example system, according to anillustrative embodiment.

FIG. 1E is a perspective view of an example system, according to anillustrative embodiment.

FIG. 1F is a perspective view of an example system, according to anillustrative embodiment.

FIG. 1G is a perspective view of an example system, according to anillustrative embodiment.

FIG. 2A is a perspective view of an example system, according to anillustrative embodiment.

FIG. 2B is a schematic diagram of an example system, according to anillustrative embodiment.

FIG. 3 is a functional block diagram of an example system, according toan illustrative embodiment.

FIG. 4 depicts a scene, according to an illustrative embodiment.

FIG. 5A depicts four scenes, according to an illustrative embodiment.

FIG. 5B depicts four scenes, according to an illustrative embodiment.

FIG. 5C depicts four scenes, according to an illustrative embodiment.

FIG. 6 depicts a scene, according to an illustrative embodiment.

FIG. 7 depicts four scenes, according to an illustrative embodiment.

FIG. 8 depicts a scene, according to an illustrative embodiment.

FIG. 9 depicts a scene, according to an illustrative embodiment.

FIG. 10 is a flowchart of an example method, according to anillustrative embodiment.

DETAILED DESCRIPTION

Example methods and systems are described herein. It should beunderstood that the words “example,” “exemplary,” and “illustrative” areused herein to mean “serving as an example, instance, or illustration.”Any embodiment or feature described herein as being an “example,” being“exemplary,” or being “illustrative” is not necessarily to be construedas preferred or advantageous over other embodiments or features. Theexample embodiments described herein are not meant to be limiting. Itwill be readily understood that the aspects of the present disclosure,as generally described herein, and illustrated in the figures, can bearranged, substituted, combined, separated, and designed in a widevariety of different configurations, all of which are explicitlycontemplated herein.

I. Overview

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

An imaging system may capture information about a scene and display alive view representation of the scene to a user in real time. In someimaging systems, the live view representation may include moreinformation or different information than what the unaided human eye maybe able to sense. Thus, the live view representation may provideenhanced vision to users.

In an example embodiment, the system may include a display, atouch-based control interface, and a control system. The display may beconfigured to provide, among other information, a live viewrepresentation to a user in real time. The live view representations maybe based on image data from one or more image-capture devices.

The control system may be configured to generate the live viewrepresentation based on the image data from the one or moreimage-capture devices. For example, the control system may receive imagedata from a video camera. The control system may apply two or moreeffects to the video stream. The control system may also receive acontrol input, such as from the touch-based control interface. Inresponse to receiving the control input, the application of the two ormore effects may be initiated, adjusted, modified, or removed from thelive view representation. The control input may alternatively oradditionally be received from other types of control interfaces. Forexample, the system may include a voice-based interface or agesture-based interface.

Upon receipt of the control input, two or more effects may be applied tothe live view representation in real time. For example, a control input,such as from a user touching a touch-sensitive surface, may initiate twoor more effects, which may be applied to the live view representation tovarying degree based the control input. Furthermore, the control inputmay persist after the two or more effects are initially applied. In oneembodiment, the control input may change based on, for example, a changein the input to the touch-sensitive surface. In response, the degree orother characteristics of the two or more effects may be adjusted.

Furthermore, a user may be able to control the impact of two or moreeffects on the live-view representation with respect to one another. Forexample, a user may interact with a two-dimensional touch-sensitivesurface to apply a first effect and a second effect to the live-viewrepresentation. The degree or extent of the respective effects may bebased on the user's finger position along perpendicular axes on thetouch-sensitive surface. In other words, in response to changing theuser's finger position along an axis, the first effect may be changed.In response to changing the user's finger position along a perpendicularaxis, the second effect may be changed. Other means of modifying oneeffect with respect to another are contemplated herein.

The two or more effects may include, but are not limited to, aslow-motion effect, a speed-up effect, a bokeh effect, a high-dynamicrange effect, and a hyperspectral effect. Other effects are possible.

In one embodiment, a user may be wearing a head-mountable device with anattached camera, touch-sensitive surface, and a display. The user may beattending a soccer game and a display of the head-mountable device mayinitially provide a live view representation via the display similar towhat the user might normally see in his/her field of view. In additionto a “normal” live view representation, a user interface may bepresented to the user. For example, a row of icons, buttons, text,and/or shapes could appear in a portion of the live view representation.

The user may initiate a first effect by, for example, pushing on thetouch-sensitive surface in a location related to one of the userinterface buttons. In response to the control input, the live viewrepresentation may appear to the view as a slowed-down representation ofthe real world. For example, a soccer ball flying through the air couldappear to the user as though it were moving slower than normal. Whilethe first effect is still initiated, the user may initiate a secondeffect. In other words, the user may touch the touch-sensitive surfacein order to select a particular displayed object in the live viewrepresentation. The user may select the soccer ball, for example. Inresponse to the second control input, a halo or highlight could be addedto the live view representation of the ball. Other combinations ofeffects are possible.

It should be understood that the above embodiments, and otherembodiments described herein, are provided for explanatory purposes, andare not intended to be limiting.

II. Illustrative Systems

The disclosure describes a touch-based system and user interface forcontrolling and presenting two or more real-time vision enhancementeffects in one device. Specifically, the disclosed system may, inresponse to a touch-based control input, apply two or more effects to alive-view representation. For example, in response to a control input, amagnification and slow-motion effect may be applied to a live-viewrepresentation in real time.

Additionally, a user interface may provide access to multiple effectsfrom a single live view interface. The user may switch between effectsor combine effects in real time.

Systems and devices in which example embodiments may be implemented willnow be described in greater detail. In general, an example system may beimplemented in or may take the form of a wearable computer (alsoreferred to as a wearable computing device). In an example embodiment, awearable computer takes the form of or includes a head-mountable device(HMD).

An example system may also be implemented in or take the form of otherdevices, such as a mobile phone, among other possibilities. Further, anexample system may take the form of non-transitory computer readablemedium, which has program instructions stored thereon that areexecutable by at a processor to provide the functionality describedherein. An example system may also take the form of a device such as awearable computer or mobile phone, or a subsystem of such a device,which includes such a non-transitory computer readable medium havingsuch program instructions stored thereon.

An HMD may generally be any display device that is capable of being wornon the head and places a display in front of one or both eyes of thewearer. An HMD may take various forms such as a helmet or eyeglasses. Assuch, references to “eyeglasses” or a “glasses-style” HMD should beunderstood to refer to an HMD that has a glasses-like frame so that itcan be worn on the head. Further, example embodiments may be implementedby or in association with an HMD with a single display or with twodisplays, which may be referred to as a “monocular” HMD or a “binocular”HMD, respectively.

FIG. 1A illustrates a wearable computing system according to an exampleembodiment. In FIG. 1A, the wearable computing system takes the form ofa head-mountable device (HMD) 102 (which may also be referred to as ahead-mounted display). It should be understood, however, that examplesystems and devices may take the form of or be implemented within or inassociation with other types of devices, without departing from thescope of the invention. As illustrated in FIG. 1A, the HMD 102 includesframe elements including lens-frames 104, 106 and a center frame support108, lens elements 110, 112, and extending side-arms 114, 116. Thecenter frame support 108 and the extending side-arms 114, 116 areconfigured to secure the HMD 102 to a user's face via a user's nose andears, respectively.

Each of the frame elements 104, 106, and 108 and the extending side-arms114, 116 may be formed of a solid structure of plastic and/or metal, ormay be formed of a hollow structure of similar material so as to allowwiring and component interconnects to be internally routed through theHMD 102. Other materials may be possible as well.

One or more of each of the lens elements 110, 112 may be formed of anymaterial that can suitably display a projected image or graphic. Each ofthe lens elements 110, 112 may also be sufficiently transparent to allowa user to see through the lens element. Combining these two features ofthe lens elements may facilitate an augmented reality or heads-updisplay where the projected image or graphic is superimposed over areal-world view as perceived by the user through the lens elements.

The extending side-arms 114, 116 may each be projections that extendaway from the lens-frames 104, 106, respectively, and may be positionedbehind a user's ears to secure the HMD 102 to the user. The extendingside-arms 114, 116 may further secure the HMD 102 to the user byextending around a rear portion of the user's head. Additionally oralternatively, for example, the HMD 102 may connect to or be affixedwithin a head-mounted helmet structure. Other configurations for an HMDare also possible.

The HMD 102 may also include an on-board computing system 118, an imagecapture device 120, a sensor 122, and a finger-operable touch pad 124 ora touch-based control interface. The on-board computing system 118 isshown to be positioned on the extending side-arm 114 of the HMD 102;however, the on-board computing system 118 may be provided on otherparts of the HMD 102 or may be positioned remote from the HMD 102 (e.g.,the on-board computing system 118 could be wire- or wirelessly-connectedto the HMD 102). The on-board computing system 118 may include aprocessor and memory, for example. The on-board computing system 118 maybe configured to receive and analyze data from the image capture device120 and the finger-operable touch pad 124 (and possibly from othersensory devices, user interfaces, or both) and generate images foroutput by the lens elements 110 and 112.

The image capture device 120 may be, for example, a camera that isconfigured to capture still images and/or to capture video. In theillustrated configuration, image capture device 120 is positioned on theextending side-arm 114 of the HMD 102; however, the image capture device120 may be provided on other parts of the HMD 102. The image capturedevice 120 may be configured to capture images at various resolutions orat different frame rates. Many image capture devices with a smallform-factor, such as the cameras used in mobile phones or webcams, forexample, may be incorporated into an example of the HMD 102.

Further, although FIG. 1A illustrates one image capture device 120, moreimage capture device may be used, and each may be configured to capturethe same view, or to capture different views. For example, the imagecapture device 120 may be forward facing to capture at least a portionof the real-world view perceived by the user. This forward facing imagecaptured by the image capture device 120 may then be used to generate anaugmented reality where computer generated images appear to interactwith or overlay the real-world view perceived by the user.

The sensor 122 is shown on the extending side-arm 116 of the HMD 102;however, the sensor 122 may be positioned on other parts of the HMD 102.For illustrative purposes, only one sensor 122 is shown. However, in anexample embodiment, the HMD 102 may include multiple sensors. Forexample, an HMD 102 may include sensors 102 such as one or moregyroscopes, one or more accelerometers, one or more magnetometers, oneor more light sensors, one or more infrared sensors, and/or one or moremicrophones. Other sensing devices may be included in addition or in thealternative to the sensors that are specifically identified herein.

The finger-operable touch pad 124 or touch-based control interface isshown on the extending side-arm 114 of the HMD 102. However, thefinger-operable touch pad 124 may be positioned on other parts of theHMD 102. Also, more than one finger-operable touch pad may be present onthe HMD 102. The finger-operable touch pad 124 may be used by a user toinput commands. The finger-operable touch pad 124 may sense at least oneof a pressure, position and/or a movement of one or more fingers viacapacitive sensing, resistance sensing, or a surface acoustic waveprocess, among other possibilities. The finger-operable touch pad 124may be capable of sensing movement of one or more fingerssimultaneously, in addition to sensing movement in a direction parallelor planar to the pad surface, in a direction normal to the pad surface,or both, and may also be capable of sensing a level of pressure appliedto the touch pad surface. In some embodiments, the finger-operable touchpad 124 may be formed of one or more translucent or transparentinsulating layers and one or more translucent or transparent conductinglayers. Edges of the finger-operable touch pad 124 may be formed to havea raised, indented, or roughened surface, so as to provide tactilefeedback to a user when the user's finger reaches the edge, or otherarea, of the finger-operable touch pad 124. If more than onefinger-operable touch pad is present, each finger-operable touch pad maybe operated independently, and may provide a different function.

In a further aspect, HMD 102 may be configured to receive user input invarious ways, in addition or in the alternative to user input receivedvia finger-operable touch pad 124. For example, on-board computingsystem 118 may implement a speech-to-text process and utilize a syntaxthat maps certain spoken commands to certain actions. In addition, HMD102 may include one or more microphones via which a wearer's speech maybe captured. Configured as such, HMD 102 may be operable to detectspoken commands and carry out various computing functions thatcorrespond to the spoken commands.

As another example, HMD 102 may interpret certain head-movements as userinput. For example, when HMD 102 is worn, HMD 102 may use one or moregyroscopes and/or one or more accelerometers to detect head movement.The HMD 102 may then interpret certain head-movements as being userinput, such as nodding, or looking up, down, left, or right. An HMD 102could also pan or scroll through graphics in a display according tomovement. Other types of actions may also be mapped to head movement.

As yet another example, HMD 102 may interpret certain gestures (e.g., bya wearer's hand or hands) as user input. For example, HMD 102 maycapture hand movements by analyzing image data from image capture device120, and initiate actions that are defined as corresponding to certainhand movements.

As a further example, HMD 102 may interpret eye movement as user input.In particular, HMD 102 may include one or more inward-facing imagecapture devices and/or one or more other inward-facing sensors (notshown) sense a user's eye movements and/or positioning. As such, certaineye movements may be mapped to certain actions. For example, certainactions may be defined as corresponding to movement of the eye in acertain direction, a blink, and/or a wink, among other possibilities.

HMD 102 also includes a speaker 125 for generating audio output. In oneexample, the speaker could be in the form of a bone conduction speaker,also referred to as a bone conduction transducer (BCT). Speaker 125 maybe, for example, a vibration transducer or an electroacoustic transducerthat produces sound in response to an electrical audio signal input. Theframe of HMD 102 may be designed such that when a user wears HMD 102,the speaker 125 contacts the wearer. Alternatively, speaker 125 may beembedded within the frame of HMD 102 and positioned such that, when theHMD 102 is worn, speaker 125 vibrates a portion of the frame thatcontacts the wearer. In either case, HMD 102 may be configured to sendan audio signal to speaker 125, so that vibration of the speaker may bedirectly or indirectly transferred to the bone structure of the wearer.When the vibrations travel through the bone structure to the bones inthe middle ear of the wearer, the wearer can interpret the vibrationsprovided by BCT 125 as sounds.

Various types of bone-conduction transducers (BCTs) may be implemented,depending upon the particular implementation. Generally, any componentthat is arranged to vibrate the HMD 102 may be incorporated as avibration transducer. Yet further it should be understood that an HMD102 may include a single speaker 125 or multiple speakers. In addition,the location(s) of speaker(s) on the HMD may vary, depending upon theimplementation. For example, a speaker may be located proximate to awearer's temple (as shown), behind the wearer's ear, proximate to thewearer's nose, and/or at any other location where the speaker 125 canvibrate the wearer's bone structure.

FIG. 1B illustrates an alternate view of the wearable computing deviceillustrated in FIG. 1A. As shown in FIG. 1B, the lens elements 110, 112may act as display elements. The HMD 102 may include a first projector128 coupled to an inside surface of the extending side-arm 116 andconfigured to project a display 130 onto an inside surface of the lenselement 112. Additionally or alternatively, a second projector 132 maybe coupled to an inside surface of the extending side-arm 114 andconfigured to project a display 134 onto an inside surface of the lenselement 110.

The lens elements 110, 112 may act as a combiner in a light projectionsystem and may include a coating that reflects the light projected ontothem from the projectors 128, 132. In some embodiments, a reflectivecoating may not be used (e.g., when the projectors 128, 132 are scanninglaser devices).

In alternative embodiments, other types of display elements may also beused. For example, the lens elements 110, 112 themselves may include: atransparent or semi-transparent matrix display, such as anelectroluminescent display or a liquid crystal display, one or morewaveguides for delivering an image to the user's eyes, or other opticalelements capable of delivering an in focus near-to-eye image to theuser. A corresponding display driver may be disposed within the frameelements 104, 106 for driving such a matrix display. Alternatively oradditionally, a laser or LED source and scanning system could be used todraw a raster display directly onto the retina of one or more of theuser's eyes. Other possibilities exist as well.

The camera 120 may be a part of an imaging subsystem that may beconfigured to capture visual information from a field of view of camera120. Camera 120 may be configured to capture images at a high framerate, such as 240 Hz. Other frame rates are possible. Camera 120 mayalso be configured to capture images at a number of different framerates. The visual information from camera 120 is processed by aprocessor (not shown) and may be presented on lens elements 110 and/or112 in real time. The type and degree of effect application to the liveview representation is controlled by users in real time, via thefinger-operable touch pad 118.

Camera 120 may represent one, two, or more image capture devicesconfigured to operate in various modes. For example, when capturingimages for the slow motion effect, only one camera operating at highframe rate is required. Frames (captured at high frame rate) may bebuffered in memory; and a program running on the system picks the rightframe to display at a proper time and at a proper frame rate, accordingto user interaction via the finger-operable touch pad 118.

The one or more image-capture devices could include, but should not belimited to, cameras, sensors, imaging systems, or photosensors. Theimage-capture devices may be configured to capture images of a field ofview. The image-capture devices may be configured to detect wavelengthsof light at least partially within the visible light spectrum.Additionally or alternatively, the image-capture devices may beconfigured to capture images in the infrared spectrum, or at otherwavelengths and/or within other wavelength bands.

When capturing image data for the low light effect, camera 120 mayrepresent a RGB camera and a black and white camera runningsimultaneously. The black and white camera captures less noisy imageswith no color, and RGB camera captures color images with relatively morenoise. Images from the black and white and RGB cameras may be combinedinto clean RGB images on-the-fly, which may be displayed instantly viathe lens elements 110 and/or 112. The user may control the level ofbrightness and/or color saturation by simple user interactions in realtime via the finger-operable touch pad 118.

User interaction in real time via the finger-operable touch pad 118 mayalternatively or additionally control how many cameras and/or whichcamera(s) may operate.

FIG. 1C illustrates another wearable computing system according to anexample embodiment, which takes the form of an HMD 152. The HMD 152 mayinclude frame elements and side-arms such as those described withrespect to FIGS. 1A and 1B. The HMD 152 may additionally include anon-board computing system 154 and an image capture device 156, such asthose described with respect to FIGS. 1A and 1B. The image capturedevice 156 is shown mounted on a frame of the HMD 152. However, theimage capture device 156 may be mounted at other positions as well, ormay be embedded into or otherwise attached to the frame.

As shown in FIG. 1C, the HMD 152 may include a single display 158 whichmay be coupled to the device. The display 158 may be formed on one ofthe lens elements of the HMD 152, such as a lens element described withrespect to FIGS. 1A and 1B, and may be configured to overlaycomputer-generated graphics in the user's view of the physical world.The display 158 is shown to be provided in a center of a lens of the HMD152, however, the display 158 may be provided in other positions, suchas for example towards either the upper or lower portions of thewearer's field of view. The display 158 is controllable via thecomputing system 154 that is coupled to the display 158 via an opticalwaveguide 160.

FIG. 1D illustrates another wearable computing system according to anexample embodiment, which takes the form of a monocular HMD 172. The HMD172 may include side-arms 173, a center frame support 174, and a bridgeportion with nosepiece 175. In the example shown in FIG. 1D, the centerframe support 174 connects the side-arms 173. The HMD 172 does notinclude lens-frames containing lens elements. The HMD 172 mayadditionally include a component housing 176, which may include anon-board computing system (not shown), an image capture device 178, anda button 179 for operating the image capture device 178 (and/or usablefor other purposes). Component housing 176 may also include otherelectrical components and/or may be electrically connected to electricalcomponents at other locations within or on the HMD. HMD 172 alsoincludes a BCT 186.

The HMD 172 may include a single display 180, which may be coupled toone of the side-arms 173 via the component housing 176. In an exampleembodiment, the display 180 may be a see-through display, which is madeof glass and/or another transparent or translucent material, such thatthe wearer can see their environment through the display 180. Further,the component housing 176 may include the light sources (not shown) forthe display 180 and/or optical elements (not shown) to direct light fromthe light sources to the display 180. As such, display 180 may includeoptical features that direct light that is generated by such lightsources towards the wearer's eye, when HMD 172 is being worn.

The HMD 172 may also include a finger-operable touch pad 182, which maybe similar or identical to the finger-operable touch pad described andshown in FIG. 1A-1C.

In a further aspect, HMD 172 may include a sliding feature 184, whichmay be used to adjust the length of the side-arms 173. Thus, slidingfeature 184 may be used to adjust the fit of HMD 172. Further, an HMDmay include other features that allow a wearer to adjust the fit of theHMD, without departing from the scope of the invention.

FIGS. 1E to 1G are simplified illustrations of the HMD 172 shown in FIG.1D, being worn by a wearer 190. As shown in FIG. 1F, BCT 186 is arrangedsuch that when HMD 172 is worn, BCT 186 is located behind the wearer'sear. As such, BCT 186 is not visible from the perspective shown in FIG.1E.

In the illustrated example, the display 180 may be arranged such thatwhen HMD 172 is worn, display 180 is positioned in front of or proximateto a user's eye when the HMD 172 is worn by a user. For example, display180 may be positioned below the center frame support and above thecenter of the wearer's eye, as shown in FIG. 1E. Further, in theillustrated configuration, display 180 may be offset from the center ofthe wearer's eye (e.g., so that the center of display 180 is positionedto the right and above of the center of the wearer's eye, from thewearer's perspective).

Configured as shown in FIGS. 1E to 1G, display 180 may be located in theperiphery of the field of view of the wearer 190, when HMD 172 is worn.Thus, as shown by FIG. 1F, when the wearer 190 looks forward, the wearer190 may see the display 180 with their peripheral vision. As a result,display 180 may be outside the central portion of the wearer's field ofview when their eye is facing forward, as it commonly is for manyday-to-day activities. Such positioning can facilitate unobstructedeye-to-eye conversations with others, as well as generally providingunobstructed viewing and perception of the world within the centralportion of the wearer's field of view. Further, when the display 180 islocated as shown, the wearer 190 may view the display 180 by, e.g.,looking up with their eyes only (possibly without moving their head).This is illustrated as shown in FIG. 1G, where the wearer has movedtheir eyes to look up and align their line of sight with display 180. Awearer might also use the display by tilting their head down andaligning their eye with the display 180.

In another embodiment, the system may take the form of other deviceswith a display or screen, such as mobile phones, tablets, PCs, etc. Forexample, as depicted in FIG. 2A, system 200 may configured like asmartphone or tablet. System 200 may include a body 202, a main button210, a secondary button 208, a first camera 204, a second camera 212,and a display 206. The display 206 may optionally include atouch-sensitive surface.

For example, display 206 may provide a live view representation of afield of view based on image data from the first camera 204 and/or thesecond camera 212. A third camera 214 may be included. In someembodiments, multiple cameras may provide stereoscopic imaging,multispectral imaging, or other capabilities. A control system (e.g. aprocessor, not shown) may be configured to generate the live viewrepresentation and receive a control input corresponding to two or moreeffects. The control input may be entered via a touch-sensitive surface,the main button 210, and/or the secondary button 208. The control systemmay be further configured to respond to the control input by applyingthe effect to the live view representation in real time.

FIG. 2B is a schematic block diagram of an illustrative system 220.System 220 may represent a monocular or binocular optical systemviewable by a user. System 220 may include optical elements such aseyepiece 222, which may also represent an ocular lens. System 220 mayalso include an objective lens 224, and other elements 226. The otherelements 226 may include, for example, processors and/or sensors. Thesystem 220 may include components similar to a telephoto camera. Theoptical elements of system 220 may share a common optical axis 238.Alternatively, the optical elements need not share the same opticalaxis. System 220 may include an alternative light source or flash 228and/or a wide field of view camera 230. System 220 may include a powerbutton 232, a mode switcher 234, and a main control 236. In someembodiments, system 220 may be approximately 3 inches long and 1 inch indiameter. Other sizes and form factors of system 220 are possible.

FIG. 3 is a schematic block diagram of an illustrative system 300. Thesystem 300 may include a display subsystem 310, an imaging subsystem320, a user interface subsystem 330, a control system 340, acomputational subsystem 350, a computer readable medium 360, and programinstructions 370. System 300 may include more or fewer subsystems and/orcomponents.

Image capture and real-time display may be performed computationally.Thus, various algorithms may be utilized to make transitions between theapplied effects smooth. For example, when transitioning out of theslow-motion effect, the computational subsystem 350 and/or the displaysubsystem 310 may gradually reduce the rate of slow-down and/or speed upthe live-view representation so as to appear smooth and/or moreaesthetically pleasing to a user.

The user interface subsystem 330 may generally include a power button, amode switch/button, and/or one or more controllers to allow users toturn on/off the device, switch modes, and control the level of visionenhancement. The user interface subsystem 330 may include atouch-sensitive surface (e.g. touch pad). A small display panel or aportion of the main user display may be provided to inform the user ofthe current state of the system and/or effect applied. The userinterface subsystem 330 may include a touch pad that need not be part ofthe display. In other words, the user interface subsystem 330 mayinclude a touch-sensitive surface distinct from the display.

The user interface subsystem 330 may be configured to accept a controlinput, such as a signal from a touch pad or other touch-sensitivesurface. Many control inputs are possible, but single fingertouch/swipe, double finger touch/swipe, multi-finger touch, and holdsare all contemplated within the scope of the disclosure. Additionally,control inputs may include other ways of forming symbols, shapes, orletters via the touch-sensitive surface.

Although a touch-based control interface is disclosed within the contextof user interface subsystem 330, other types of control interfaces arecontemplated. For example, the user interface subsystem 330 may includea voice-based or a gesture-based control interface. Correspondingly,control inputs generated by user interface subsystem 330 may betriggered or initiated by voice or gesture commands from a user.

Upon receipt of the control input, the control system 340 may add,subtract, and/or modify two or more effects applied the live-viewrepresentation. Furthermore, the control system 340 may modify at leasta first and second effect with respect to one another, which may affectthe live-view representation. For example, a user may initiate aslow-motion effect. In such a scenario, in response to the user moving afinger upward on the touch-sensitive surface, the control system maymodify the slow-motion effect with respect to a speed-up effect oranother effect. Other combinations of effects are possible.

The computational subsystem 350 may include processors, memory, storage,and communication means (like WiFi and Bluetooth). The computationalsubsystem 350 may control the flow of information from the othersubsystems; it may process images/video from the imaging subsystem 320.The computational subsystem 350 may (according to effect(s) applied anduser interaction) process and fuse images from different sensors; and itfeeds processed images/video to the display subsystem 310 instantly. Insome implementations, at least some of the computational subsystem 350may include cloud computing resources. In other words, the computationalsubsystem 350 may include processing capabilities that are remote fromthe display subsystem 310 and/or the imaging subsystem 320.

Generally, systems and methods disclosed here may include theapplication of various effects to a live view representation viewablefrom a display. FIG. 4 depicts a scenario 400 according to anillustrative embodiment. Scenario 400 could include a soccer playerpreparing to kick a ball toward a soccer goal and goalkeeper. Scenario400 may also represent a live-view representation viewable by a user ofthe HMD or another display type. As such, the live view representationmay include a user interface. The user interface may include icons,words, and/or other indicators (e.g. lights and/or sounds) that may beassociated with effects applicable to the live view representation. Forexample, the icons may include a slow motion icon 402, a zoom icon 404,a select icon 406, a bokeh icon 408, and an other icon 410. The liveview representation may further include a soccer player 412, a ball 414,a goalkeeper 416, and a net 420.

User interaction via the touch-based control interface may cause aneffect to be selected. Such a selection may cause the corresponding iconto be highlighted, illuminated, underlined, enlarged, or otherwiseidentified. For example, the slow motion icon 402 may have a lightedoutline as depicted in FIG. 4.

FIG. 5A depicts four successive images that occur after scenario 400.Each of the images may represent a live view representation viewable bya user via a display. For example, images 500 and 505 may depict thesoccer player preparing to kick the ball toward the goalkeeper. Image510 may include the soccer player making contact with the ball. Image515 may include the ball in the air moving toward the goalkeeper.

FIG. 5B depicts four successive images that occur after image 515. Forexample, images 520, 525, 530, and 535 may depict the ball moving closerto the goalkeeper. FIG. 5C depicts four successive images that occurafter image 535. Images 540, 545, and 550 may depict the ball moving yetcloser to the goalkeeper. Image 555 may include the goalkeeper making asave on the ball with his/her hands.

Although FIGS. 5A-C depict twelve images, more or less images may becaptured by the system over the representative time frame. For example,images may be captured (and possibly displayed via the display) at 250Hz or at another frequency.

If the slow motion mode is activated, for instance in response to a userinteraction via the touch-based control interface, the live viewrepresentation may include images from FIGS. 5A-C. However, thedisplayed images may appear “slower” than compared to the real timescene. For example, if the images from FIGS. 5A-C were initiallycaptured 100 milliseconds apart, the live view representation viewableby a user may display the representative images with 200 millisecondsbetween representative images. The live view representation may thusappear to a user as though the ball is moving more slowly toward thegoalkeeper. In some embodiments, the live view representation mayinclude other actual captured frames or interpolated frames so as tomake the slow motion sequence appear more smoothly.

In response to a user interaction via the touch-based control interface,the slow motion effect may stop immediately, returning the live viewrepresentation to real time, or the live view representation maytransition from slow motion to real time. For instance, a user mayrelease his/her finger from the touch-based control interface. Inresponse, the live view representation may “speed up” in order to catchup with the real time scene.

For example, a user may touch the touch-based control interface so as tostart the slow motion effect. The live view representation may bedisplayed to the user as a “slowed down” version of the real world. Theuser may subsequently stop touching the touch-based control interface.In response, the live view representation may display images at a higherrate than the original capture rate so as to “catch up” the live viewrepresentation with the real time scene.

As an alternative way to control the slow-motion effect, a user fingerposition may be changed on the touch-sensitive surface so as to adjustthe speed of the live-view representation. In other words, if forexample a user moves her finger upward, the live-view representation mayspeed up to actual speed or even faster than actual speed.Alternatively, if a user moves her finger down on the touchpad, thelive-view representation may slow down further. Other user interactionsare possible so as to control the slow-motion effect via the touch-basedcontrol interface.

Another effect applicable to the live view representation may include amagnification of a scene or a portion of a scene. FIG. 6 depictsscenario 600 according to an illustrative embodiment. Scenario 600 mayrepresent the live view representation viewable by a user via a display.The live view representation may include a user interface. The userinterface may include icons or other indicators associated with effectsthat may be applied to the live view representation. For example, theicons may include a slow motion icon 602, a zoom icon 604, a select icon606, a bokeh icon 608, and an other icon 610. The live viewrepresentation may further include a soccer player 612, a ball 614, agoalkeeper 616, and a net 620.

A user may interact with the touch-based control interface to initiatethe magnification mode. For example, the user may touch the touch-basedcontrol interface at a location corresponding to the magnification icon.Other user interactions to initiate the magnification mode are possible.In response to initiating magnification mode, the magnification icon 602may be illuminated, highlighted, underlined, or otherwise specificallyidentified on the live view representation.

While in magnification mode, the live view representation, or a portionof the live view representation may appear magnified. The user mayspecify a portion of the live view representation to be magnified. Forexample, the user may push on the touch-based control interface at alocation corresponding to the ball 614 on the live view representation.In response, a magnification inset 608 may appear. The magnificationinset 608 may include a magnified view of the area near the portion ofthe live view representation specified by the user. As shown in FIG. 6,the soccer player's feet and the ball 614 may appear magnified withinthe magnified inset 608.

The user may interact further with the touch-based control interface tofurther control the magnification mode. For example, the user may pushharder on a button or touch-sensitive surface so as to magnify themagnified areas further, while pushing less hard on the button ortouch-sensitive surface may magnify those areas to a lesser extent.

The slow motion modes and magnification modes may be initiatedsimultaneously. That is, a user may initiate slow motion andmagnification modes at the same time via the touch-based controlinterface. Alternatively, the slow motion mode and magnification modesmay be initiated at different times, but the modes may be both appliedto the live view representation concurrently. For instance, the user mayinitiate slow motion mode, and while the live view representation is“slowed down”, the user may subsequently initiate magnification mode to“zoom into” the soccer ball or other portions of the live viewrepresentation. Other combinations of the slow motion and magnificationmodes are possible.

Another effect applicable to the live view representation may include anobject or area selection effect. FIG. 7 depicts four successive images700, 705, 710, and 715 based on scenario 500. A user may initiate theobject or area selection effect, for example by touching the touch-basedcontrol interface. In response, the user interface may change byhighlighting, underlining, illuminating, or otherwise specificallyidentifying the select icon 702. Further, the user interface mayhighlight, underline, illuminate, recolor, fill, or otherwisespecifically identify the object or area selected by the user.Generally, the selection mode may change the live view representationmake it easier to identify or track the selected object or area. Withregard to FIG. 7, the ball 704 may be circled so as to make it easier tofollow as it moves through the air toward the goal keeper.

In some embodiments, the slow motion effect may be combined with theobject or area selection effect. For example, upon pressing thetouch-based control interface, a slow-motion effect may be initiated onthe live view representation, which may slow down fast-moving objects orindividuals. The user may then move his/her finger on the touch-basedcontrol interface to cause a selection box to move around on thelive-view representation. Once the selection box is moved to “select”the desired object or individual, the user may release the button, thelive-view representation may return to real time and the selection boxmay persist, tracking the desired object or individual.

In alternative embodiment, a background/foreground defocus effect may beutilized. FIG. 8 depicts an illustrative embodiment that includesscenario 800. Scenario 800 may represent a live view representationsimilar to that of scenario 400, as illustrated and described inreference to FIG. 4. Scenario 800 may include user interface elements,including various icons, such as a slow motion icon 802, a zoom icon804, a select icon 806, a bokeh icon 808, and an other icon 810. Thelive view representation may also include a soccer player 812, a ball814, a goalkeeper 816, a net 820, and background scenery 822.

A user may initiate a background/foreground defocus effect by, forexample, interacting with the touch-based control interface. Inresponse, the user interface may cause the bokeh icon 808 to beilluminated or otherwise highlighted. Further, in response, the userinterface may apply a narrow depth of field effect to the live viewrepresentation of the scenario 800. The narrow depth of field effect mayappear via the live view representation similar to a an image capturedby a single lens reflex camera with a wide aperture lens. In otherwords, soccer player 812 and the ball 814 may be in focus, howeverelements of the live view representation in the foreground (e.g. net 820and goalkeeper 816) or in the background (background scenery 822) mayappear out of focus.

The background/foreground defocusing or blurring effect, also calledbokeh, may be controllable based on, for instance, a user fingerposition on the touch-sensitive surface. The elements in or out of focusmay depend on a user interaction. For example, the user may interactwith the touch-based control interface to initiate bokeh mode.Subsequent user interactions (e.g. moving finger up, down, left, orright on touch-based control interface) may change the focus depthand/or the apparent depth of field on the live view representation.Other ways of controlling the degree of the effect on the live-viewrepresentation are possible.

In some embodiments, a face substitution effect may be applied to thelive-view representation. In such a scenario, a user may be able toinitiate a “face swap” of a person in his/her field of view and thenchange the face based on the user finger position on the touch-sensitivesurface. The substituted face may track the person's body so as toappear like the substituted face is a part of the person's body. Thesubstitution of other body parts, such as noses, mouths, eyes, etc. arecontemplated as well. Further, the substitution of entire bodies orother objects is considered within the scope of the disclosure.

FIG. 9 depicts an illustrative embodiment as shown by scenario 900.Scenario 900 may represent a live view representation viewable by a uservia a display. Scenario 900 may include a user interface, which mayfurther include effects icons, such as an other icon 902. A user mayinitiate a “jersey swap” effect by, for example, interacting with atouch-based control interface. In response, the user interface mayhighlight, illuminate, underline, or otherwise specifically identify theother icon 902 or any other icon associated with the initiated effect.In scenario 900, the other icon 902 may represent a “jersey swap”effect. As such, once “jersey swap” mode is initiated by the user, thelive view representation may be adjusted such that soccer player 904 andgoalkeeper 906 appear to have the opposing team's jersey. Other “swap”effects or replacement effects are considered within the scope of thisdisclosure.

Another effect applicable to the live view representation may includecolor enhancement/shifting. For example, a user may be color blind. Insuch a scenario, a color shifting effect may be applied to change greenobjects in the environment so that they appear to the user as adifferent color in the live-view representation. Other colors, or rangesof colors, in the environment may be shifted to appear as a differentcolor or different color range in the live-view representation. Inanother example, color saturation may be increased in response tochanges in color in the environment. For instance, amedical-practitioner user may wish to observe an individual'scirculation near the skin surface. By applying the color saturationeffect, very slight color changes may be enhanced to appear as moresubstantial color changes in the live-view representation.

High Dynamic Range, or HDR, may be applied to the live viewrepresentation as a further effect. For example, while in HDR mode, thedynamic range of the live-view representation may appear more compressedthan normal. The user may be able to control the degree of HDRcompression by moving his/her finger on the touch-sensitive surface orchanging pressure on the button. Other related effects, such asbrightness and contrast enhancement are possible.

In some embodiments, sensors configured to image various spectral bandsmay be utilized as an effect on the live view representation. Forinstance, in an IR/RGB mode, the user may be able to initiate and adjusta live-view representation wherein an infrared image and an RGB image ofthe field of view are combined in a single, real-time, display. Otherspectral bands are possible. For example, UV/RGB, UV/X-ray, etc.

In another embodiment, alternate viewing modes may represent one or moreeffects. For example, a cat-vision effect may be controlled by a userthat may allow a user to observe the environment similar to a cat. Otheranimal (e.g. dog, fish, bird, etc.) vision modes are contemplated withinthe scope of this disclosure.

In yet another embodiment, a multispectral imaging effect may be appliedto the live-view representation. For example, a false color map may beused so that a user could observe a representation of light outside thevisible spectrum. Other substitution maps are possible. For example, asymbolic (e.g. cross-hatching) map may be used.

In a further embodiment, a “kid-mode” effect may be utilized to assistwith or teach photographic technique. For example, a user may initiatekid mode by touching a touch-sensitive surface. The imaging subsystem320, or camera, may be provided to a child or someone learning aboutphotography. The child and/or the person learning about photographymight wave the camera around in an arbitrary manner. The imagingsubsystem may detect that a suitable subject is properly composed withinthe field of view and trigger a still image or video clip to berecorded.

III. Illustrative Methods

FIG. 10 is a flowchart of a method 1000 for providing a live-viewrepresentation of image data from one or more image-capture devices andapplying two or more effects to the live-view representation, where theeffects are controllable in real time via a touch-based controlinterface. Method 1000 could be carried out with the systems describedand illustrated in FIGS. 1A-9. Blocks need not be carried out in theorder presented. Other systems and steps may be utilized to carry outmethod 1000. Block 1010 includes providing a live view representation ofa field of view to a display. The live view representation is based onimage data from one or more image-capture devices. Block 1020 includesreceiving a control input for two or more effects via a touch-basedcontrol interface. Block 1030 includes for each of the two or moreeffects, responding to receipt of the control input for the effect viathe touch-based control interface by applying the effect to the liveview representation in real time. The two or more effects may include,but are not limited to, the effects described above.

IV. Illustrative Non-Transitive Computer Readable Medium

Some or all of the functions described above and illustrated in FIG. 10may be performed by a computing device in response to the execution ofinstructions stored in a non-transitory computer readable medium. Thenon-transitory computer readable medium may be, for example, a randomaccess memory (RAM), a read-only memory (ROM), a flash memory, a cachememory, one or more magnetically encoded discs, one or more opticallyencoded discs, or any other form of non-transitory data storage. Thenon-transitory computer readable medium may also be distributed amongmultiple data storage elements, which may be remotely located from eachother. The non-transitory computer readable medium could be the computerreadable medium 360 as described and illustrated in FIG. 3. Thecomputing device that executes the stored instructions may include thecomputational subsystem 350 as described and illustrated in FIG. 3.Additionally or alternatively, the computing device may include anothercomputing device, such as a server in a server network.

The non-transitory computer readable medium may store instructionsexecutable by a computing device (e.g. computational subsystem 350 asdescribed in reference to FIG. 3) to cause the computing device toperform any of the functions described herein. The functions may takethe form of program instructions 370 as described and illustrated inFIG. 3.

In one example, the functions include causing a live view representationof a field of view to be provided to a display. The live viewrepresentation is based on image data from one or more image-capturedevices. The functions further include receiving a control input for twoor more effects via a touch-based control interface. The functions yetfurther include, for each of the two or more effects, responding toreceipt of the control input for the effect via the touch-based controlinterface by causing the effect to be applied to the live viewrepresentation in real time.

V. Conclusion

Where example embodiments involve information related to a person or adevice of a person, some embodiments may include privacy controls. Suchprivacy controls may include, at least, anonymization of deviceidentifiers, transparency and user controls, including functionalitythat would enable users to modify or delete information relating to theuser's use of a product.

Further, in situations where embodiments discussed herein collectpersonal information about users, or may make use of personalinformation, the users may be provided with an opportunity to controlwhether programs or features collect user information (e.g., informationabout a user's medical history, social network, social actions oractivities, profession, a user's preferences, or a user's currentlocation), or to control whether and/or how to receive content from thecontent server that may be more relevant to the user. In addition,certain data may be treated in one or more ways before it is stored orused, so that personally identifiable information is removed. Forexample, a user's identity may be treated so that no personallyidentifiable information can be determined for the user, or a user'sgeographic location may be generalized where location information isobtained (such as to a city, ZIP code, or state level), so that aparticular location of a user cannot be determined. Thus, the user mayhave control over how information is collected about the user and usedby a content server.

The particular arrangements shown in the Figures should not be viewed aslimiting. It should be understood that other embodiments may includemore or less of each element shown in a given Figure. Further, some ofthe illustrated elements may be combined or omitted. Yet further, anillustrative embodiment may include elements that are not illustrated inthe Figures.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the true scopebeing indicated by the following claims.

The invention claimed is:
 1. A system comprising: a mobile computingdevice, wherein the mobile computing device comprises: one or moreimage-capture devices; a display, wherein the display is configured todisplay a live view representation of a field of view based on imagedata from the one or more image-capture devices; and a touch-basedcontrol interface; and a control system configured to: generate the liveview representation of the field of view based on image data from theone or more image-capture devices; display, in the live viewrepresentation of the field of view, computer-generated graphicscomprising a plurality of icons, wherein each icon of the plurality oficons is representative of a corresponding effect; receive, via thetouch-based control interface, control input indicative of a selectionof a first icon from the plurality of icons; in response to receivingcontrol input indicative of the selection of the first icon, produce, inthe live view representation in real-time, a first effect thatcorresponds to the selected first icon; receive, via the touch-basedcontrol interface, control input indicative of a selection of a secondicon from the plurality of icons; in response to receiving control inputindicative of the selection of the second icon, produce, in the liveview representation in real-time, a second effect that corresponds tothe selected second icon, wherein the first and second effects areproduced concurrently in the live view representation in real-time;receive, via the touch-based control interface, control input indicativeof a first adjustment to the first effect being produced, wherein thefirst adjustment is based on a first variation of touch on thetouch-based control interface; in response to receiving control inputindicative of the first adjustment, adjust, in accordance with the firstadjustment that is based on the first variation of touch on thetouch-based control interface, the first effect being produced; receive,via the touch-based control interface, control input indicative of asecond adjustment to the second effect being produced, wherein thesecond adjustment is based on a second variation of touch on thetouch-based control interface, and wherein the second variation of touchis different from the first variation of touch; and in response toreceiving control input indicative of the second adjustment, adjust, inaccordance with the second adjustment that is based on the secondvariation of touch on the touch-based control interface, the secondeffect being produced.
 2. The system of claim 1, wherein the one or moreimage-capture devices comprises a sensor configured to capture images ofthe field of view.
 3. The system of claim 2, wherein the sensor isconfigured to capture images of the field of view based on visiblelight.
 4. The system of claim 2, wherein the sensor is configured tocapture images of the field of view based on infrared light.
 5. Thesystem of claim 1, wherein the touch-based control interface comprisesone or more of the following: (i) a touch-sensitive surface and (ii) abutton.
 6. The system of claim 1, wherein the first effect comprises oneof the following effects: (i) a slow-motion effect, (ii) a speed-upeffect, (iii) a bokeh effect, and (iv) a high dynamic range effect. 7.The system of claim 1, wherein the control system being configured toproduce the first effect comprises the control system being configuredto produce the first effect while receiving, via the touch-based controlinterface, control input associated with the first effect, and whereinthe control system is further configured to: detect that control inputassociated with the first effect is no longer being received via thetouch-based control interface; and in response to detecting that controlinput associated with the first effect is no longer being received viathe touch-based control interface, stop producing, in the live viewrepresentation in real-time, the first effect that corresponds to theselected first icon.
 8. The system of claim 1, wherein the controlsystem is further configured to: receive, via the touch-based controlinterface, control input indicative of a further adjustment to the firsteffect; and in response to receiving control input indicative of thefurther adjustment to the first effect, adjust the first effect inaccordance with the further adjustment and with respect to the secondeffect.
 9. The system of claim 1, wherein the first effect comprises ahyperspectral effect and the second effect comprises a slow-motioneffect, a speed-up effect, a bokeh effect, or a high dynamic rangeeffect.
 10. A method comprising: capturing a live view representation ofa field of view based on image data from one or more image-capturedevices of a mobile computing device; providing the live viewrepresentation from the one or more image-capture devices to a displayof the mobile computing device; displaying, in the live viewrepresentation of the field of view, computer-generated graphicscomprising a plurality of icons, wherein each icon of the plurality oficons is representative of a corresponding effect; receiving via atouch-based control interface of the mobile computing device, controlinput indicative of a selection of a first icon from the plurality oficons; in response to receiving control input indicative of theselection of the first icon, producing, in the live view representationin real-time, a first effect that corresponds to the selected firsticon; receiving, via the touch-based control interface, control inputindicative of a selection of a second icon from the plurality of icons;in response to receiving control input indicative of the selection ofthe second icon, producing, in the live view representation inreal-time, a second effect that corresponds to the selected second icon,wherein the first and second effects are produced concurrently in thelive view representation in real-time; receiving, via the touch-basedcontrol interface, control input indicative of a first adjustment to thefirst effect being produced, wherein the first adjustment is based on afirst variation of touch on the touch-based control interface; inresponse to receiving control input indicative of the first adjustment,adjusting, in accordance with the first adjustment that is based on thefirst variation of touch on the touch-based control interface, the firsteffect being produced; receiving, via the touch-based control interface,control input indicative of a second adjustment to the second effectbeing produced, wherein the second adjustment is based on a secondvariation of touch on the touch-based control interface, and wherein thesecond variation of touch is different from the first variation oftouch; and in response to receiving control input indicative of thesecond adjustment, adjusting, in accordance with the second adjustmentthat is based on the second variation of touch on the touch-basedcontrol interface, the second effect being produced.
 11. The method ofclaim 10, wherein producing the first effect comprises producing thefirst effect while receiving, via the touch-based control interface,control input associated with the first effect, the method furthercomprising: detecting that control input associated with the firsteffect is no longer being received via the touch-based controlinterface; and in response to detecting that control input associatedwith the first effect is no longer being received via the touch-basedcontrol interface, stopping to produce, in the live view representationin real-time, the first effect that corresponds to the selected firsticon.
 12. The method of claim 10, further comprising: receiving, via thetouch-based control interface, control input indicative of a furtheradjustment to the first effect; and in response to receiving controlinput indicative of the further adjustment to the first effect,adjusting the first effect in accordance with the further adjustment andwith respect to the second effect.
 13. A non-transitory computerreadable medium having stored therein instructions executable by amobile computing device to cause the mobile computing device to performfunctions comprising: generating a live view representation of a fieldof view based on image data from one or more image-capture devices ofthe mobile computing device; causing the live view representation to beprovided to a display of the mobile computing device; displaying, in thelive view representation of the field of view, computer-generatedgraphics comprising a plurality of icons, wherein each icon of theplurality of icons is representative of a corresponding effect;receiving, via a touch-based control interface of the mobile computingdevice, control input indicative of a selection of a first icon from theplurality of icons; in response to receiving control input indicative ofthe selection of the first icon, producing, in the live viewrepresentation in real-time, a first effect that corresponds to theselected first icon; receiving, via the touch-based control interface,control input indicative of a selection of a second icon from theplurality of icons; in response to receiving control input indicative ofthe selection of the second icon, producing, in the live viewrepresentation in real-time, a second effect that corresponds to theselected second icon, wherein the first and second effects are producedconcurrently in the live view representation in real-time; receiving,via the touch-based control interface, control input indicative of afirst adjustment to the first effect being produced, wherein the firstadjustment is based on a first variation of touch on the touch-basedcontrol interface; in response to receiving control input indicative ofthe first adjustment, adjusting, in accordance with the first adjustmentthat is based on the first variation of touch on the touch-based controlinterface, the first effect being produced; receiving, via thetouch-based control interface, control input indicative of a secondadjustment to the second effect being produced, wherein the secondadjustment is based on a second variation of touch on the touch-basedcontrol interface, and wherein the second variation of touch isdifferent from the first variation of touch; and in response toreceiving control input indicative of the second adjustment, adjusting,in accordance with the second adjustment that is based on the secondvariation of touch on the touch-based control interface, the secondeffect being produced.